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ABSTRACT 



This paper makes the case that the standards -based reform 
movement is too limited an approach to rely on in educational reform, and 
that there is a set of hard facts that must be faced if there is to be any 
significant improvement in student achievement. These facts are the 
attitudes, practices, and conditions that are so embedded in culture and 
personal experience that they escape serious attention. Without an 
understanding of the preconditions that have an impact on improvement in 
academic rigor and student achievement, there is little basis for serious 
consideration of new ways and means of raising educational success. Tests are 
being used to hold students and schools accountable for standards before much 
needed teaching capacity is built. In addition, tests are not aligned to the 
standards, and tests are not aligned to the curriculum. The standards -based 
reform movement can only succeed if policymakers and educators pay attention 
to the larger societal context and the environment in which change is to be 
implemented. Issues that must be considered in this context include student 
behavior, weak signals that academic achievement is important, the 
acknowledgement of the importance of sources of learning outside the school, 
and the appropriate use of educational technology. It is not appropriate to 
lessen efforts to implement the standards-based reform agenda, but this alone 
will not be enough for significant improvement. (SLD) 
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Preface 



As I write this preface, the Congress is moving 
towards approval of the Elementary and 
Secondary Education Act, which will be the 
primary mechanism for charting federal involve- 
ment in our nation’s schools. Arnong the 
major issues being debated is the role of 
increased student testing as a tool for ensuring 
educational quality. As an assessment 
organization, Educational Testing Service has a 
vested interest in how these matters play out. 
Indeed, as an organization that has at the core 
of its mission serving as the leader in educational 
assessment, we try to do more than simply watch 
policies unfold — we try to provide useful 
information to policymakers that can help them 
in their deliberations. 

We do this by supporting the kind of policy 
work that is exemplified in this report by Paul 
Barton. We don’t undertake this work to sell 
tests. Rather, we do it because we take seriously 
our role in opening up educational opportunity 
for all. Policy research that illuminates condi- 
tions, challenges conventional wisdom, unpacks 
the complexity of issues, and provides salient 
and credible information on politically charged 
matters can play a significant role in helping 
policymakers make informed decisions. 

Facing the Hard Facts is a synthesis of a 
decade of work and illuminates some very criti- 
cal issues involved in transforming American 
education, issues that are not at the center of 
the current educational debate and developing 
legislation. If we pay attention to Barton’s claims, 
we will make progress on issues that open up 
opportunities for all students. If we only focus 



on the test scores, and not on the alignment of 
educational policies and practices that influ- 
ence those scores, we will do nothing more than 
blame those who have not benefited from the 
system. If there isn’t sufficient order in schools 
and classrooms to ensure a reasonable learning 
environment, then it matters not what curricula 
are followed or what textbooks are purchased. 
If we don’t, as a society, begin to value academic 
achievement more consistently and encourage 
practices that develop social capital for all, then 
we will continue to expand the divide of the 
haves and have-nots. Finally, if we do not take 
advantage of the power of technology in all 
schools, then we will have wasted enormous 
financial resources as we erroneously equate 
access to hardware with access to ideas. 

Barton raises a number of key points, but 
even these are not an exhaustive list. School 
finance and teacher quality are but two addi- 
tional issues that will continue to challenge our 
values and assumptions about the importance 
of education, the nature of social responsibil- 
ity and accountability, and fairness. 

I hope that you as readers will find this 
report as provocative as I do. I also encourage 
you to visit our website at www.ets.org/ 
research/ pic , where you can download addi- 
tional Policy Information Center reports. 

Drew H.Gitomer 
Senior Vice President, 
Statistics and Research 
Educational Testing Service 
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wnsiderable momentum has been gained in 
education reform under the umbrella of standards- based reform. 



There is general agreement that this includes, 
at a minimum, setting standards for what con- 
tent should be taught and what performance 
students should be expected to demonstrate, 
and for creating assessments aligned to those 
standards. The approach seems inherently 
logical, and a general consensus has developed 
that this is the way to proceed. I am part of 
that consensus. However, I also believe that 
the formulation is incomplete, and that imple- 
mentation in many places (but by no means 
all) has strayed from the path. 

I argue that this standards-based reform 
movement is too limited an approach to rely 
on, and that there is a set of hard facts we 
must face if we are to see any really significant 
improvement in student achievement. The 
nation has set ambitious goals, and the effort 
to achieve them must be equally ambitious. 
The analysis presented here does not address 
ways to restructure schools or schooling, 
though this important debate, subject as it is 
to polarization, should continue. It is my 
belief that the matters discussed in this report, 
while they have yet to take center stage, are 
mostly within the broad center of viewpoints. 
My intent is to bring information and per- 
spectives to bear that have not yet been given 
serious attention. 

Of course, while standards-based reform 
is the driving force today, with a focus on 
defining and measuring outcomes, policy 
makers and educators do not all view the 



current reform movement as standards-based 
reform and nothing else. There are efforts in 
various places and levels of intensity that 
reflect a broader view of education reform. 
But school standards, accountability, and test- 
ing have occupied center stage. 

It is not surprising that basic change in 
institutions is difficult as societies and cultures 
strive to continue to exist, and to recreate 
themselves. Education is very much engaged 
in passing on the civilization to succeeding 
generations, and formal schooling is part of 
that process. As a result, we see a demand for 
teaching values we fear the younger genera- 
tion has lost or may lose. We see parents look- 
ing at what goes on in schools and comparing 
their childrens experiences to the experiences 
they had; they are often suspicious of large 
changes. Due to a myriad of conflicting cul- 
tural attitudes, we see considerable toleration 
of disorder in the schoolroom, although com- 
mon sense tells us it harms learning; boys and 
girls will be boys and girls, won’t they? And 
new uniforms for the marching band may well 
get more support than making sure every stu- 
dent has a textbook. 

If we truly want succeeding generations 
to learn significantly more than past ones, we 
must, as a society, in a sense, pull ourselves up 
by our bootstraps. Knowing this, we examine 
our shoelaces and other seemingly mundane 
matters, such as cutting classes, getting 
employers to ask for school transcripts, and 
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reading to young children in the home. And 
then there are those computers that, so far, 
have not realized their promise. At any one 
time, a few proposals and issues dominate the 
political and professional discussion. But as 
the buzz goes on, there remain those attitudes, 
practices, and conditions that are so embed- 
ded in our culture and personal experience 
that they escape our serious attention. These 
are facts that must be faced. 

This report does not recommend a lot of 
specific policy prescriptions. It is intended to 
provide the understanding necessary to for- 
mulate those prescriptions. Without an 
understanding of the many preconditions that 
impact significant improvement in academic 
rigor and student achievement, there is little 
basis for. serious consideration of new ways 
and new means of raising educational success. 



er|c 
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The Test is Not the Treatment 



eople who have committed themselves to 
the standards'based reform approach recognize that standardized testing is a very 
important part of that approach. 



They also recognize the role testing should 
play in our total approach to education 
reform. It turns out, though, that in imple- 
menting reform, many states have somehow 
come to view the test itself as the treatment, 
rather than as a way to monitor whether 
reform measures are actually producing 
results. At the same time, tests are increasingly 
being used for high-stakes purposes — the 
means of holding schools, teachers, and 
students accountable. All this is creating 
a backlash in many states that is driven 
by students, parents, and the public, and 
which is threatening forward progress in rais- 
ing student achievement. 

There are those who would protest most 
all uses of standardized achievement tests 
when important consequences are attached to 
the results. But when used properly, the right 
tests can be an important element in raising 
student achievement. In a recent report, the 
National Research Council put it this way: 

When tests are used in ways that meet 
relevant psychometricy legal ^nd educational 
standardsy students' scores provide important 
information thaty combined with information 
from other sourcesy can lead to decisions that 
promote learning and equality of opportunity. ^ 



There are many considerations involved 
in the use of standardized tests. The points 
discussed below have to do with two princi- 
pal aspects. One is that we are using tests to 
hold students and schools accountable for 
standards before building much needed teach- 
ing capacity. The other is that tests that mea- 
sure student achievement are being misused. 
Of course, whole books and thick reports are 
written about tests and their uses. We will 
confine this discussion to several important 
ways testing can go astray through inappro- 
priate use. We need to heed the warning in 
the National Research Council report, cited 
above: “When test use is inappropriate... It 
can undermine the quality of education and 
the equality of opportunity.” 

Aligning Tests to Curriculum 

Students should be tested for what they 
are taught; that is, tests should be aligned to 
curriculum. The standards-based reform 
movement that began in the 1980s contem- 
plated a process that would set challenging 
content standards based on broad agreement 
as to what students should know and be able 
to do. Next, performance standards were to 



* Heuberc, J. R, & Hauser, R. M. (eds.). (1999). High Stakes: Testing for Tracking, Promotion, and Graduation, (National 
Research Council Report). Washington, DC: National Academy Press. 
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be set to determine how much of this content 
students should be held accountable for mas- 
tering. Finally, after content standards had 
been translated into curriculum actually in use 
in the classroom, tests were to be developed 
to help determine whether students had 
learned what they were supposed to learn. 

By now, all states have by and large 
established the content standards, although 
reviewers have judged them to be uneven in 
quality with regard to such things as how spe- 
cific and rigorous they are. Next, most or 
many states jumped quickly into administer- 
ing standardized tests on a large-scale basis, 
and established passing scores, often with fed- 
eral help and prodding. But the intervening 
steps of changing curriculum, improving the 
teaching materials, and preparing teachers to 
deliver new curriculum were often skipped 
over, or at least not completed. 

Despite skipping these vital steps, the 
results of these tests frequently have impor- 
tant consequences. They are placed on indi- 
vidual student records; they are used to judge 
the performance of schools and their teach- 
ers; and they are the bases for allocating 
resources; and sometimes for determining 
whether students pass, fail, go to summer 
school, or graduate. As far as the experts in 
educational measurement are concerned, this 
is an inappropriate use of tests; for those who 
are not experts, it simply defies common sense. 



We are beginning to get information 
based on careful observation of what is 
happening in practice in some uses of large- 
scale testing. A recent study issued by the 
Council of Chief State School Officers, 
entitled Alignment of Science and Mathemat- 
ics Standards in Four States found that “align- 
ment between assessments and standards var- 
ied across grade levels, content areas, and states 
without any discernible pattern.” The Coun- 
cil also has underway an 1 1 -state study called 
the Survey of the Enacted Curriculum^ which 
is being carried out in collaboration with the 
Wisconsin Center for Education Research. A 
recently issued summary of this project con- 
cluded that, “even where widely disseminated 
and used by teachers, standards do not pro- 
vide curricula for teaching in classrooms.”^ In 
eight of the states studied, the survey found 
that in fourth-grade mathematics and eighth- 
grade science, “less than half the intersections 
of content topics [being taught in the class- 
room]... were in common with the assessment 
items found on the state test.” 

In other words, we see tests not aligned 
to the standards, and tests not aligned to the 
curriculum. When students and parents see 
tests that do not reflect what is being taught, 
they lose confidence in the reform movement. 
The testing results tell as much about the 
incompleteness of implementing standards- 
based reform as they do student, teacher, and 
school performance. 



- Webb, N. L. (2000). 

^May 2000. Council of Chief State School Officers. 
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Using Multiple Criteria for High- 
Stakes Decisions 

When making important decisions about 
students, teachers, and schools, good tests 
used appropriately may help, but standard- 
ized tests should never be the sole basis for 
such decisions. This is one matter that experts 
in the testing business agree on, whether those 
experts are reputable testing companies, edu- 
cational measurement professionals, or 
researchers in the area of testing and psycho- 
metrics. Standards for test use issued by the 
American Psychological Association, the 
American Educational Research Association, 
and others frown on such a practice. A 1999 
report issued by the National Academy of 
Science,^ had this to say: “No single instru- 
ment can serve all purposes well. Assessment 
should involve a range of strategies appropri- 
ate for inferences relevant to individual stu- 
dents, classrooms, schools, districts, and 
states.” The National Research Council 
report, cited earlier, gives this admonition: 

A test score, like other sources of information, 
is not exact. It is an estimate of the student's 
understanding or mastery at a particular 
time. Therefore, high-stakes educational 
decisions should not he made solely or auto- 
matically on the basis of a single test score 
but should abo take other relevant informa- 
tion into account. 



It is understandable that many would 
want to rely exclusively on test scores. There 
is a desire for criteria external to the classroom 
— to show objectivity when important deci- 
sions are made. Test scores are relatively inex- 
pensive. And they provide a specific number, 
which gives an aura of precision. But people 
who know that a single number is not an 
adequate basis for such decisions stress that 
human judgment is still required. Other cri- 
teria are also important, such as teacher judg- 
ment, grades, homework, classroom partici- 
pation, and tests given by teachers. If we 
ignore the professional judgments of teach- 
ers, we send a message to those who teach, 
and those who might want to be teachers, that 
their judgments are not valued or trusted. 
Good judgment in the classroom will decline, 
not increase. 

If standardized tests are to be used to 
judge schools, they must measure the gain in 
knowledge during a school year, not the abili- 
ties and knowledge acquired during a lifetime. 
The tests we now use, with only a few excep- 
tions, measure what students know and can 
do at a point in time. If it is a math test at the 
end of the eighth grade, it captures what the 
student has learned in the prior seven grades, 
in the family, and in the community. Whats 
more, it reflects the student s cognitive devel- 
opment — the result of nutrition and intel- 
lectual stimulation provided generally in the 
early childhood period. How do we identify 



^ Elmore, R., & Rorhman, R. (eds.). (1999). Testings Teaching and Learning: A Guide for States and School Districts: 
Washington, DC: National Academy of Science. 
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the addition to knowledge a school achieved 
during the eighth grade? And if we look at 
test scores of a school s eighth graders over 10 
years, how do we factor in how much of the 
change, up or down, is due to changing 
demographics of the student body and how 
rriuch is due to what the school did? 

Data from the National Assessment of 
Educational Progress (NAEP) illustrate what 
a test oi gain in knowledge — as opposed to a 
test of level of knowledge — can tell us about 
schools. In 1992 and 1996, of the 37 states 
taking the assessment in both years, Maine 
had the highest fourth- and eighth-grade 
NAEP scores in mathematics, and Arkansas 
had the lowest. However, the gain in these 
NAEP scores — from 1 992 when the students 
were in the fourth grade to 1996 when the 
students were in the eighth grade — was 52 
scale points in both Maine and Arkansas. 
Arkansas, of course, is one of the poorest states 
in the nation in terms of per capita income. 
Yet, the schools in Arkansas added as much 
math knowledge in those four years as 
did Maine. In terms of state testing, only 
Tennessee currently uses such a gain -score test- 
ing system, and William Sanders has pio- 
neered the development and use of these 
value-added measures.^ 

It isnt either/or in testing; we need to 
know both. The test of total knowledge tells 
how society as a whole is doing in youth 
development; the test of gain in knowledge. 



from the beginning to the end of the year, gets 
closer to revealing how well schools do in add- 
ing knowledge. From a technical standpoint, 
measuring gain has its own set of difficulties, 
which need to be overcome. 

The large-scale use of standardized test- 
ing has become a mainstay of education 
reform efforts in the United States, and in 
holding schools accountable for results. If edu- 
cation reform will stand or fall on how well 
we use standardized testing, we better take a 
close look at what we are doing and get it right. 
Its a matter of achieving success, and of being 
fair to students, parents, teachers, and the edu- 
cation enterprise generally. 

We argue here for putting standards-based 
reform back on the track it started on. But 
even if we do, the reform movement can suc- 
ceed only if policymakers and educators pay 
attention to the larger societal context and 
environment in which change is to be imple- 
mented. This larger context is addressed in the 
next sections of this report. 



^ This analysis is provided in Growth in School^ by Paul Barton and Richard Coley, published by Educational Testing 
Service in 1998. 
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Student Behavior: Stani 


)ARDS 

F. 


TO Increase Achievement 

several years now, the focus of attention 



on student behavior has been on instances of extreme violence. 



As important as this is, and as correct as 
parents and students are to be concerned, these 
are relatively rare events; 99 percent of all 
homicides involving children happen away 
from school. 

What is not rare, is in fact widespread 
and is growing — not declining — is student 
behavior that is disruptive to the learning 
environment of the school and of the class- 
room. This situation has attracted almost no 
national attention, other than the campaign 
of the American Federation of Teachers 
begun by the late Albert Shanker, and is not 
addressed in the standards-based reform 
movement. But ask any teacher and you will 
be told how much order and discipline mat- 
ter to learning. 

The Behaviors 

School behavior problems are not uni- 
form across states, and they vary as well among 
children from different socioeconomic classes. 
Exposure to learning disruption also falls 
unequally by race and ethnic group. The kinds 
of behavior that are problematic vary highly, 
from fighting, to verbal abuse of teachers, to 
cutting classes. Problem behaviors tend to fall 
into two categories: those like cutting classes, 
that impact the achievement of the student 



practicing the behavior, and those that impact 
the climate of the entire classroom or school, 
and thus the achievement of all students. 

While nationally, just 1 4 percent of eighth 
graders said, in 1996, that physical conflicts 
were moderate or serious problems in their 
schools, the percentage was higher in indi- 
vidual states: 44 percent in Hawaii, 38 per- 
cent in Maryland, and 35 percent in Georgia. 
According to data from the NAEP mathemat- 
ics assessment, just nine percent of students 
felt unsafe or very unsafe in the nations schools 
as a whole in 1996, but 20 percent of stu- 
dents in the District of Columbia, and 14 
percent in Mississippi, South Carolina, and 
Florida, felt this way. Fewer students, just five 
percent, felt unsafe in North Dakota. 

As Figure 1 demonstrates, trends track- 
ing problem behaviors from 1991 to 1997 are 
generally unfavorable. While there was a 
decline in alcohol use during this period, and 
no statistically significant changes were 
reported for some discipline issues, behaviors 
that worsened*" were; 

■ tardiness 

■ absenteeism or cutting classes 

■ drug use 

■ verbal abuse of teachers 

■ sale of drugs on school grounds 



^ That is, the change was statistically significant. 
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Figure 1 : Percentage of Public School Principals Reporting that 
Various Discipline Issues Were Serious or Moderate Problems in 
Their High Schools, 1990-91 and 1996-97 



Student tardiness 



150 



Student tobacco use j 



Student absenteeism/ 
class-cutting 

Student alcohol use 

Student drug use 

Physical conflicts 
among students 

Teacher absenteeism 

Robbery over $10 



1 17 

J 17 
I 17 



114 
■ 15 



20 



Verbal abuse 
of teachers 
Vandalism of 
school property 

Trespassing 




20 * 



Racial tensions 



I I5 



Student possession i 1 3 

of weapons mM2 

Sale of drugs l 12 
on school grounds ^■^^^^^■ 10 * 

Physical abuse n 1 
of teachers ■ 1 



Teacher alcohol □ 1 
or drug use <.5 



^ Not surveyed in 1991 
Gangs 



20 



27 



140 



139 



]37* 



48 



52* 



36* 



□ 1990- 1991 
■ 1996- 1997 



* Indicates statistically significant difference from 1991. 



30 40 50 60 

Percentage 



70 



Source: NCES, 1998 



For a slightly different time period, 1 989 
to 1995, we know that there was an increase 
in “violent victimization” of 12- to 19-year- 
old students from 3.4 to 4.2 percent.^ The 
same group of students also reported an 
increase in the presence of street gangs in 
schools — 28 percent reported their presence 
in 1995, compared to 15 percent in 1989. 
However, for Black students the statistic rose 



from 20 to 35 percent, and for Hispanic stu- 
dents, from 32 to 50 percent. Gangs at school 
are becoming a widespread phenomenon in 
the United States, and their presence is being 
felt unequally. 



^ The U.S. Department of Justice’s School Crime Supplement, 1989 and 1995. 
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It Matters 

Some of these behaviors have obvious 
impacts on teaching and learning. Class cut- 
ting, absenteeism, and tardiness reduce learn- 
ing time. Drugs and alcohol are a drag on 
health, on studying, on attention, and on 
attendance. Physical conflict and the presence 
of gangs cause fear in going to and from school 
and at the school itself And that fear is ris- 
ing. In 1989, six percent of all 12- to 19-year- 
old students feared attack or harm at school, 
and four percent were afraid while going to 
and from school. By 1995, those figures had 
risen to nine and seven percent, respectively. 
The proportion who fear harm at school is 
higher for Black students: one in six Black 
students felt fear in suburban schools.^ Fear 
and learning are not good companions. 

As a matter of science, though, little has 
been done to enable us to say precisely which 
behaviors harm achievement, or how much. 
An ambitious 1978 study measured problem 
behaviors, but not achievement. Recently, 
Harold Wenglinsky of Educational Testing 
Service (ETS®) analyzed the results of a study 
begun in 1988 called the National Education 
Longitudinal Study. Using that study s 



achievement test in combination with infor- 
mation regarding student behaviors elicited 
by questionnaires answered by students, teach- 
ers, and parents, Wenglinsky found a 
relationship between student behaviors and 
achievement. He summarized the results 
this way: 

The frequency of serious and nonserious 
offenses is negatively related to academic 
achievement in all four subject areas studied 
— mathematics y reading, science, and social 
studies... The frequency of drug offenses is 
negatively related to academic achievement 
in mathematics and science, but not in social 
studies and reading.^ 

Another likely impact of growing disrup- 
tion in the school environment is its effect on 
the supply of people willing to enter and con- 
tinue in the teaching profession. Many prob- 
lem behaviors are directed toward teachers. A 
review of what research exists on this relation- 
ship led this author to conclude, in a book 
recently published by The Century Fund, 
that discipline does make a difference in teach- 
ers leaving. 



® Kaufman, R, Chen, X., Choy, S. R, Chandler, K. A., Chapman, C. D., Rand, M, R. & Ringel, C, (1998). Indicators of 
School Crime and Safety, 1998 (NCES 98-25 1/NCJ- 17221 5). Washington, DC: U.S. Department of Education and Justice 
^ Barton, R, Coley, R, & Wenglinsky, H, (1998). Order in the Classroom: Violence, Discipline, and Student Achievement 
(ETS Rolicy Information Report). Rrinceton: NJ: Educational Testing Service. 

A Notion at Risk: Preserving Public Education as an Engine for Social Mobility, in a chapter by this author titled “Unequal 
Learning Environments: Discipline That Works,” edited by Richard Kahlenberg and commissioned and published by The 
Century Fund (formerly the 20th Century Fund) in September of 2000. 
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A Variety of Approaches 

In the last couple of decades, a variety of 
serious attempts to improve student behaviors 
have been directed at the more serious prob- 
lems of bringing weapons to school, at aggres- 
sive behaviors more generally, and at more gar- 
den variety forms of disorder and disruption. 
Some of these efforts involved evaluations of 
their effectiveness, conducted with varying 
degrees of rigor, and with generally positive 
results. Others used “before and after” com- 
parisons, or anecdotal evidence of improve- 
ment, to measure their success. A review of 
these practices, as well as the results of those 
evaluations that exist, are provided in the Cen- 
tury Fund book, referenced above. Order in 
the Classroom y also previously cited, presents a 
study of trends in behaviors and different 
approaches to modifying them. 

Student behaviors that reduce the effec- 
tiveness of teaching and impact student learn- 
ing do not have to be accepted as inevitable. 
However, to reduce the incidence of these 
behaviors, choices must be made among a wide 
variety of approaches for which solid scientific 
evidence of effectiveness is not always avail- 
able (much like the case in the education field 
generally). Experience must be combined with 
good judgment. Likely, the best solution is not 
a simple matter of using a single approach, but 
the combination that works best and fits best 
with the culture of the school and community. 
Examples of these approaches include: 



■ Statewide 'zero tolerance^^ legislatioHy as in 
West Virginia and Texas. 

■ Alternative schools. Expelling students is a 
more frequent means of enforcing zero 
tolerance policies. However, there are not 
enough of these “alternative schools” to 
accommodate the students who need 
them. 

■ New district level "code of behaviors'^ as in 
Cincinnati, Ohio. 

■ New disciplinary authority for teachers in 
collective bargaining contracts, as in Min- 
neapolis, Minnesota. 

■ Character educationy now supported with 
Federal grants and undergoing evaluation 
in each state receiving those grants. 

■ Teaching conflict resolutiony as in the Day- 
ton, Ohio program called the “Positive 
Adolescent Choices Training Program.” 

■ Strict school security measures y such as metal 
detectors, restricted school entry, “closed 
campuses” that restrict students from leav- 
ing school, and the presence of law 
enforcement personnel in schools. (Such 
approaches were surveyed in 1997 by the 
National Center for Education Statistics.) 

■ Classroom management systems y such as one 
called “Consistency Management and 
Cooperative Discipline.” 

There is, of course, an interaction 
between student behavior and the academic 
environment. The brief review provided 
above considers how improving student 
. behavior can improve teaching and learning. 
It is likewise true that good teaching and 
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more challenging subject matter can improve 
student behavior and classroom discipline. It 
is a two-way street, and we must go down both 
sides of it. Standards of behavior are a vital 
part of standards-based reform, and reports 
of behaviors must be viewed against those 
standards. In some states, student behaviors 
are factored into accountability systems, and 
measures of school climate are presented on 
school report cards. It may be such account- 
ability that forces us to find ways to improve 
student behaviors and the learning climate in 
the school and classroom. 
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Weak Signals That Academic Achievement is Important 

Tl quality of education is at the top of 
the list of what concerns Americans these days, particularly when it comes to electing 
public ojficials. 



But when it comes down to the signals the 
individual student receives from his or her 
environment, how high is the country's goal 
for student learning? That is the question. 
True, teachers and schools share a responsi- 
bility to create the desire to learn by 
presenting challenging subject matter in the 
classroom and by using teaching skills to 
stimulate interest. But this is not the only 
source of student motivation by far. Motiva- 
tion comes from a myriad of messages a stu- 
dent receives — in the family, in the commu- 
nity, on the job, and in the culture — from 
the portrayal of learning in sitcoms to the ways 
students gain social acceptance from their 
peers. The hard fact is that American culture 
is not really a learning culture, at least not in 
the K-12 period of life. 

Signals From Parents 

Recognizing, of course, that there is a lot 
of variation among individuals from the same 
life circumstances, the shaping of the motiva- 
tion to excel in school comes from the imme- 
diate family and environment during child- 
hood. Generally, striving in school has a lot 
to do with the signals young children receive. 
I would argue that the “typical” American 
family wants to raise a fairly “typical” Ameri- 
can child — one who is popular with play- 
mates and has a lot of interests and activities. 



Yes, do your homework, but concern will 
develop if the parent thinks that the teacher is 
loading too much on the child. If the math 
grades are a litde low, it may well prompt a 
remark that math was hard for me also. Par- 
ents have a larger agenda for their children than 
studying: In a pubic opinion poll conducted 
by Gallup for Phi Delta Kappa International 
in June of 2000 {The Washington Times, 
August 23, 2000), 42 percent of parents said 
that school extracurricular activities are as 
important as academic studies. Parents, though, 
put great emphasis on getting a high school 
diploma; the diploma itself is often valued more 
than academic achievement. 

Developing student interest in learning is 
critical. In the Third International Mathemat- 
ics and Science Study (TIMSS), 51 percent of 
American eighth-grade math teachers said that 
having “uninterested students” limited their 
teaching “quite a lot” or “a great deal.” That's 
just the teacher's perspective, of course, but the 
percentages were lower in Germany (43 per- 
cent) and Japan (37 percent). 

Yes, improving education seems to be a hot 
political issue. Real estate agents will tell you 
that schools are a key consideration when 
people pick out their houses. But the Gallup 
poll director speaking in Phi Delta Kappa 
International said: 
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The notion the public is dissatisfied with its 
public schools is based on myth instead of fact... 
The data indicate that public approval of the 
public schools has trended consistently upward 
and is near its all-time high. . . And as demon- 
strated in every previous poll, the closer people 
get to the public schools [in their neighbor- 
hoods], the better they like them. 

While there is a demand for better school 
quality nationally, parents are likely to get up 
in arms if they see too much changing in their 
schools — especially if those changes upset the 
pattern of education they experienced and are 
used to. Thus, if we are to see much move- 
ment upward in academic achievement, it is 
critical that families come to believe that their 
schools must raise their academic standards 
and that their children must do much 
better academically. 

Signals From Peers 

Students feel as though they are being 
asked to commit an unnatural act if they are 
expected to strive for a goal not appreciated 
by their peers. Striving too much, or too 
obviously, may actually cause them to be less 
popular with their peers. They are often less 
than admired if they study too much. If they 
have a strong bent in a particular academic 
area, they may be thought of as geeks, par- 
ticularly if they want to talk about this 
instead of the current movie in town. If they 
are both academic achievers and not terribly 
apt socially, they may receive a worse 
label, nerds. 
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Students who do well in school, but with- 
out obvious effort, can be socially accepted; its 
OK to be smart. On the other hand, 
parents can sometimes be heard apologizing for 
a daughters success: “Well, yes, but she has to 
work hard for it.” The value placed on hard 
work varies with individuals and families, and 
with the cultures that frequently accompany 
race and ethnicity. For example, Asian Ameri- 
cans have a reputation for placing a high value 
on academic achievement. And according to 
the scholarly literature, Black youth are often 
accused of “acting White” when they work 
hard. 

Signals From Colleges 

Given that there is a high value placed on 
going to college in the United States and col- 
leges have entry requirements, college must pro- 
vide some incentive to do well in academics, at 
least during high school when students and 
families realize the time for college is looming. 
But, looking at the nation as a whole, the 
incentive to excel in school is not delivered 
strongly from the admissions requirements of 
the postsecondary system. True, for selective 
colleges, doing well is necessary for admission. 
But it is also true that only one in four 25- to 
29-year-olds has earned a bachelors degree, a 
proportion that has been a constant over most 
of the last quarter of a century, moving up a 
few percentage points only over the last two or 
three years. And it is also true that the vast 
majority of postsecondary institutions are not 
selective, or not very selective. How well you 
do in high school may determine where you 
can go to college, but it seldom determines 
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whether you can go. Didn’t learn the basics of 
good English and math in school? Well, three 
out of 10 college freshmen take remedial 
courses to prepare for real college courses. 

Signals From Employers 

How about the incentive to excel in 
school in order to get a better paying job? It is 
a frequently heard message in the United 
States that how far you go in education deter- 
mines how well you do in the employment 
world. The facts clearly support such state- 
ments. In the workforce, each advance in level 
of education is accompanied, on average, with 
an advance in success in the labor market 
(as measured by lower unemployment and 
higher pay). 

Unfortunately, there is a period from age 
17 or 18 until well into the 20s when 
this does not hold up. This finding is so 
counterintuitive, and counter to conventional 
wisdom, it is hard for many people to accept 
as true. Employers who offer what I call adult- 
type work, with good pay and fringe benefits, 
are generally loath to hire young people just 
out of high school, preferring to wait until 
they are well into their 20s — their percep- 
tion of the age of maturity. Employers who 
do hire young people at age 17, 18, or 19, 
hire them for “youth jobs,” all of which pay 
about the same and do not require much edu- 
cation. In actuality, the employment prospects 
of high school graduates with above average 
grades and test scores are no better, at these 



ages, than those with lower grades and test 
scores; high school dropouts do as well as high 
school graduates. But as these young people 
advance into their late 20s, the situation 
begins to change, and those who excel in 
school begin doing better. Employers do value 
a high school diploma. 

The summer 1999 issue of ETS Policy 
examines the labor market experience 
of youths leaving high school. Figure 2 dem- 
onstrates the difference grades make two years 
out of high school in terms of employment 
and earnings, and the data show very little 
difference in terms of the percent of young 
people employed, and no statistically signifi- 
cant differences in terms of average monthly 
earnings. A national database linking test 
scores and labor market success from age 19 
to age 31 also shows little difference in these 
early years, but it shows that the higher per- 
formers begin to pull ahead at about age 23. 

The message for young people is that 
performance in school does make a difference, 
but they may not see it for a while. Their aca- 
demic performance will also determine their 
college chances, and college will give them 
better opportunities. But it is likely a hard 
message to get across when high school stu- 
dents see what happens in the short term to 
those who graduate. They know that, by and 
large, employers do not ask to see school tran- 
scripts.’^ The incentive to achieve in high 
school based on employment success is not 
apparent enough to young people with short 
time perspectives. 



" ETS Policy Notes. (1999). “Learn More, Earn More?” Princeton, Nj: Educational Testing Service. 

‘‘ More are, though, through recent efforts of a business coalition led by the National Alliance of Business. 

: 




Facing the Hard facts • 17 



Figure 2: The Difference Grades Make: 

Employment and Earnings of Male and Female 1992 High School 
Graduates (not enrolled in school) Two Years After High School 
(1994), BY Grade Point Average 
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Strong signals to achieve are necessary if 
we are to raise academic achievement in pub- 
lic schools. The signals young people get and 
respond to come from parents, peers, teach- 
ers, counselors, colleges, and employers. 
Teachers, who want the best for their students 
and are being pushed by standards-based 
reform and accountability efforts to raise stu- 
dent achievement, deliver strong messages to 
try hard. But schools are not getting a lot of 
help from the other players in students’ lives. 
They will not get far if they have to go it alone. 

It is true that opinion polls put educa- 
tion at the top of the list of public concerns, 
at least in terms of what people want from 
political leaders. But little of this demand 
translates effectively into improving the qual- 
ity of classroom teaching, paying teachers pro- 
fessional wages, or advancing the professional 
development of the existing teaching force. A 
strong signal from the public that education 
really does matter would be to take the steps 
necessary to improve teaching and learning. 
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A Learning Policy Recognizes Sources 
OF Learning Outside the School 

T 

JL her e is nationwide consensus about the need 
to raise educational achievement, and there is widespread agreement that we are gener- 
ally on the right track in the standard- based reform movement. 



I am in total agreement with both opinions. 
But what has emerged along the road to 
achieving reform is a reluctance to examine 
all of those factors that determine student 
achievement. Our focus is on academic stan- 
dards in schools. And although this is, in part, 
as it should be, we must also work on a broader 
front if we are to raise educational achieve- 
ment significantly. This includes important 
factors beyond school doors. 

To a considerable extent, our reluctance 
to address important nonacademic factors 
stems from a fear that to consider such fac- 
tors may cause us to lose focus, and that rec- 
ognition of these factors may provide excuses 
for not raising standards and achievement. In 
fact, it is becoming popular to say “no excuses” 
— heading teachers, schools, and others who 
examine such things off at the pass. We tend 
to put considerations.of family, community, 
and economy off-limits in education-reform 
policy discussions. However, we do so at our 
peril. The seriousness of our purpose requires 
that we learn to rub our bellies and pat our 
heads at the same time. 

The research on the role these factors 
outside the school play is clear enough, 
although it tends to be expressed in research- 
ese. No matter: Plain common sense tells us 
that if parents are important, two are better 
than one; if young children are not exposed 
to reading, they will not learn to read; if 
children are not well nourished and kept in 
good health, the development of their minds 



will be held back; rich parents have more 
resources than poor parents; caring neighbors 
and a sense of community create a positive 
environment for children to grow up in; and 
parents who involve themselves in their 
childrens schools and education contribute to 
student learning. 

One way we have examined the role of 
nonschool factors in student achievement has 
been through the relationship between socio- 
economic status (family income and parent 
occupations) and school success. The first 
large-scale study of this relationship — the 
now famous Coleman Report, commissioned 
by the Federal Government and published in 
1966 — showed that variations in student 
socioeconomic status was by far the largest 
explanation for variations in student test 
scores. This nonschool factor explained much 
more of these differences than did school fac- 
tors. The Coleman data has been analyzed and 
re-analyzed. Subsequently, many other stud- 
ies have confirmed the strength of such fac- 
tors in correlations with student test scores. 
In other writing and research, James Coleman 
used the concept of “social capital” (defined 
by David Grissmer, below) to describe the role 
of socioeconomic status in student achieve- 
ment, independent of what happens in 
the schoolroom. 

It is, of course, not family income or what 
the parent does at work all day that affects 
student learning. This gross construct, socio- 
economic status, has not been decomposed 
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into those specific things that do affect 
student learning, the totality of which is also 
evidently correlated with student achieve- 
ment. We could, of course, pursue a policy of 
reducing this wide spread in family income 
in the United States, and in doing so we could 
expect to narrow the difference in student 
achievement. But while we are a nation com- 
mitted to equality of opportunity, we are not 
committed to equalizing family income. In 
fact, income has become more unequal in the 
past few decades (and is also more unequal 
than in other developed countries). To better 
inform a learning policy, we need to identify 
more specifically those factors that affect stu- 
dent development, and separate those that we 
can do something about in specific efforts 
from those that are more dependent on larger 
societal change. 

Experience in one area reassures us that we 
can improve learning outcomes with preschool 
interventions. The Head Start program of the 
1960s has been found effective in evaluations 
over several decades. There is strong circumstan- 
tial evidence that it was the investment in Head 
Start and other preschool programs in the 1960s 
that reduced the gap in scores between White 
,and minority students in the 1980s. 

The matter of “family factors” in student 
achievement was explored in 1992 in a report 
issued by the ETS Policy Information Cen- 
ter.*"^ The report identified family factors from 
the 1990 NAEP assessments and related them 



to achievement scores. For example. Figure 3 
shows, state by state, the percentage of eighth 
graders with three or more types of reading 
materials in the home, which varies from 90 
percent in North Dakota to 68 percent in 
California. The figure also shows students* 
average NAEP score in mathematics for each 
state. North Dakota is at the top in math 
achievement and Louisiana is at the bottom.*^ 
In the Summer 1993 issue oi ETS Policy 
NoteSy^^ the five family factors most strongly 
related to achievement were identified. The 
positive factors were having two parents in the 
home, reading more than two pages a day for 
school and homework, and having at least 
three types of reading materials in the home. 
The negative factors were student absentee- 
ism and excessive television watching. This 
report showed that 91 percent of the differ- 
ences among the states in test scores were 
associated with these five factors, considered 
together. Figure 4 shows how actual state 
NAEP scores compare with scores that would 
be expected based on the five factors listed 
above. We see that some higher-scoring states, 
like New Hampshire and Wyoming, did less 
well than expected based on these family 
factors, and that some lower-scoring south- 
ern states, such as Texas and Florida, did bet- 
ter than expected. 

In 2000, David Grissmer developed mea- 
sures of family and social capital and applied 
them to state NAEP achievement scores.*^ By 



An early analysis of this can be found in The Reading Report Card, 1985, Educational Testing Service. 

Barton, P. & Coley, R. (1992). Americas Smallest School: The Family (ETS Policy Information Report). Princeton, NJ: 
Educational Testing Service. 

The line (on the right hand side) that runs down the middle of the score chart shows how close the relationship is 
between the number of reading materials in the home and NAEP scores. 

ETS Policy Notes. (1993). “Angles on Math Achievement.” Princeton, NJ: Educational Testing Service. 

See Improving Student Achievement,' What State NAEP Scores Tell Us, Rand. 
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Figure 3: The Percentage of Eighth Graders with Three or More Types of 
Reading Materials in the Home and NAEP Math Proficiency, 1990 





Percent of Students with Three or More Average NAEP Math Score 

Types of Reading Materials in Honne 



Source: Americas Smallest School: The Family, (1992). (ETS Policy Information Report). Princeton, 
NJ: Educational Testing Service. 
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Figure 4: Actual and Predicted NAEP Mathematics Score by State, 1990 
(Prediction Based on School Absence, TV Watching, Reading at Home, Reading 
Materials in Home, and Presence of Two Parents in the Home) 
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family capital, Grissmer means “the charac- 
teristics within families that create higher 
achievement;” social capital refers to influences 
on achievement “through such things as peer 
effects, quality of communication and trust 
among families in communities, the safety of 
neighborhoods, and the presence of commu- 
nity institutions that support achievement.” 
Using raw NAEP scores, Grissmer asks and 
answers two questions: How much do these 
scores differ on an absolute basis, and then, 
how much do they differ among students from 
families that are similar in terms of family and 
social capital. The actual variables used by 
Grissmer were parental educational levels, fam- 
ily income, race and ethnicity, family type, 
family mobility, and two measures of socio- 
economic status. Using the raw NAEP scores, 
there was considerable variation among the 
states: “The average scores in the highest 
ranked state would be approximately between 
the 62nd and 67th percentile nationally, while 
average scores from the lowest state would be 
around the 33rd to 38th percentile. This rep- 
resents a significant variation in test scores 
among students from different states.” 

However, when he controlled for family 
and social capital characteristics, this differ- 
ence in test scores shrunk tremendously. In 
the estimates of score differences for students 
from similar families, there is a difference from 
top to bottom of only 11 or 12 national per- 
centile points. Who is near the top and who 
is near the bottom shifts considerably, as was 
the case in the ETS comparison of actual and 
predicted NAEP scores based on family 
resources and practices (Figure 4). Some 



well-off northern states near the top in the 
raw scores ended up much lower down in the 
adjusted ranking, and some southern states 
near the bottom ended up nearer the top. In 
terms of average state scores, the differences 
that various school approaches made were 
much less significant than differences due to 
family and social capital. These latter variables 
accounted for 75 percent of the differences in 
test scores. However, the Grissmer study also 
identifies education policies and practices that 
do matter. 

In the ETS Policy Information Report, 
Growth in School, cited earlier, NAEP data are 
presented based on growth in scores of 
cohorts of students from the fourth to the 
eighth grade, and trends in these cohort scores 
are compared to trends based on levels of 
knowledge . of fourth and eighth graders. 
Across about two and a half decades, level of 
knowledge rose in three out of four subjects, 
but did not rise in any of the four in terms of 
the cohorts, or in the growth in knowledge 
over four years of schooling. Therefore, the 
improvement stemmed from what was hap- 
pening before the students took the NAEP 
assessment in the fourth grade, and likely from 
changes happening outside of school before 
they entered the first grade. 

Another example of this difference 
between level and growth is shown in Figure 
5. As noted earlier, of the 37 states participat- 
ing in NAEP in both 1992 and 1996, Maine 
had the highest math scores and Arkansas had 
the lowest. But in terms of growth, the stu- 
dents in both states improved by 52 scale 
points. Arkansas is a poor state, but its 
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Figure 5: Average NAEP Mathematics Scores and Cohort Growth, 
Arkansas and Maine 
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students grew as much, over their earlier 
scores, as students in Maine. We ask, do 
schools in Maine do a better job in teaching 
mathematics from fourth grade to eighth grade 
than do schools in Arkansas? Further, can we 
raise student achievement significandy by deal- 
ing with the nonschool factors involved? 

We need a better means of tracking where 
changes in educational achievement are com- 
ing from. We need to face the hard fact that 
improving educational achievement means at- 
tending to what happens in American families 
and the effects of our social capital. Without 
doubt, in the last several decades since NAEP 
started providing measures of educational 
progress, schools have faced an uphill battle in 
terms of trends in social capital. This period 
coincides with what Francis Fukuyama called 



T/^e Great Disruption in a book by that name. 
He concluded that “beginning in about 1965, 
a large number of indicators that can serve as 
negative measures of social capital all started 
moving upward rapidly and at the same time.”'^ 
The purpose of measurement is to help 
us find those pressure points at which we can 
fashion efforts to improve student achieve- 
ment — including those that occur early in 
life and in spheres outside the school. On the 
bright side, significant efforts are being made 
in some areas, and federal and state support 
for early childhood and early literacy programs 
has grown. But education achievement indi- 
cators have hardly begun to budge. The 
socioeconomic status of young children 
remains as strong as ever a predictor of their 
relative achievement in American society. 



Fukuyama, F. (2000). The Great Disruption. New York, NY: Simon & Shuster. 
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ter of the use of computers in the classroom 




Dumb Computers and Smart People 



n a somewhat different category is the mat- 



Schools have not ignored technology. Com- 
puter use is becoming pervasive, and many 
look to this advancement as a magic bullet. 
But what has not been grasped is how much 
more we have to do. The hard fact here is how 
much more we must invest in people and in 
the educational content that gets put into the 
machines. 

This story begins in the 1930s, and it 
would be fascinating to know where it will 
end. Konrad Zuse is credited with being the 
first to use the binary system in the creation 
of a computer in Nazi Germany in 1 935 and 
1936; he was perhaps the first to have the idea 
of doing so. Yet, even as computers do ever 
more amazing things, it remains a fact that all 
they can really do is distinguish between 0 
and 1 — albeit with ever increasing speed. In 
this sense, the computer is a dumb machine, 
one that can be made by human intelligence 
to perform remarkable tasks. That said, dumb 
machine or not, the ability of Colossus to 
break the German codes created by ENIGMA 
received a lot of credit in the winning of World 
War II, and computers continue to be made 
to do things that amaze us. 

The real possibilities of computers in the 
classroom may have been imagined by a num- 
ber of the early creators of computers, but they 
were put in writing in a prophetic way by 
Christopher Evans in 1979.*^ He said, “In 



schools, computer teaching — initially for 
‘drill and practice’ — will begin to spread as 
costs spiral downwards.” Schools were indeed 
early users of computers, and at some point 
they began arriving at schools in truckloads as 
some combination of lower costs, obsolescence, 
and tax laws resulted in massive donations from 
computer makers. To a considerable extent they 
were still pretty dumb machines, lacking the 
software brains that would enable them to de- 
liver useful instruction. 

And used for drill and practice they were, 
just as had been delivered earlier by various 
versions of programmed learning. Drill and 
practice is still a principal use of computers in 
terms of instruction, as is using word process- 
ing programs to write. According to a recent 
assessment by Larry Cuban, professor of edu- 
cation at Stanford University, “When the type 
of classroom use is examined, we find that 
these powerful technologies end up being used 
most often for word processing and low-end 
applications. And this is after a decade of 
increases in access to computers, Internet 
capability, and purchases of software.” There 
are, to be sure, a variety of creative instruc- 
tional uses now employed in many classrooms, 
and schools are making investments in soft- 
ware. One step forward is that we can now 
track these developments in Education Weeks 
yearly issue of Technology Counts. 



Evans, C. (1979). The Micro Millennium. New York, NY: First Washington Square Press. 
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There is little doubt that the computer 
can be a useful tool for education. The ques- 
tions before us now are how useful, for 
exactly what purposes, and what is their role 
in relation to human teachers. Often, the 
debate about appropriate roles and uses is just 
part of the general debate about effective 
instructional strategies and pedagogies. How 
much “drill and practice” should there be — 
whether delivered through traditional 
means or through a computer? How much 
“constructivism” should there be in instruc- 
tional approaches — whether instruction is 
from the teacher or the computer? Increas- 
ingly, it seems computers can do most any- 
thing software designers want them to do, 
except supply the human element that is rec- 
ognizably important for the motivation to 
learn. Each remarkable use, such as Big Blue 
beating Karpov, reinforces the view that they 
can one day serve well in the classroom. 

The extent of the belief that computers 
can be made to deliver instruction is appar- 
ent in the conclusion that the highly regarded 
Howard Gardner has come to: The ultimate 
in approaches is one tutor to one student, and 
the computer can be used to deliver individu- 
alized instruction. He puts it this way: 

For the first time in human history, it is easy 
to envision a mass educational environment 
where instruction can be truly individual. 
Computers can be programmed so that they 
present information and interactions that are 
appropriate for each student; moreover, as 
^intelligent systems,' they can record 
approaches that have worked more or less well 







for a particular student, and readjust the cur- 
riculum and assessments accordingly. 

Gardner is quick to point out, however, 
that the human teacher is not replaceable. 
“Human beings learn, and want to learn, 
because they want to be like individuals they 
admire, who care for them, and for whom they 
care,” he states. The computer does not 
replace the teacher on the other end of the 
log, but a computer can be a huge help by 
providing access to a tutorial. 

The problem is not how smart the com- 
puter can be made to behave, but whether the 
human beings that form the system of school- 
ing can be smart enough, and motivated 
enough, to figure out how to get students to 
use computers effectively, and how to create 
access to this resource on as routine a basis as 
we now have teachers in classrooms. How does 
such a problem get resolved within the pro- 
fessional, sociocultural, bureaucratic environ- 
ment that exists within the larger governance 
system? How we will make decisions about 
computers is related to how we make other 
difficult decisions — such as how to antici- 
pate, or not anticipate, a teacher shortage due 
to the baby boom echo; how to make salaries 
competitive enough to attract capable people; 
how to get good textbooks; or how to create 
safe and orderly learning environments. These 
are all things we don’t do very well. If we can 
create the demand, and provide the resources, 
computer hardware and software systems seem 
to be in ready supply. But the hard fact is that 
we must achieve computer use in an institu- 
tional arrangement that has many difficulties 
in making such large transformations. 
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We have little scientific proof of what 
computers can accomplish in the classroom. 
Here again, judgments and deductions of 
various kinds are necessary. One large-scale 
study of mathematics achievement using the 
NAEP database found that using computers 
to teach higher order thinking skills, and pro- 
viding professional development in classroom 
computer uses to teachers, results in higher 
student achievement scores.^^ Such evaluation 
will have an important role in the choices to 
be made. 

Preparing Teachers to Use 
Computers in the Classroom 

There is a huge challenge in preparing 
teachers to use computers in the classroom. 
Several different measurement efforts disclose 
inadequate preparation of teachers now in 
the workforce. 

Schools of education now require 
instruction in technology. But Education 
Week's 1999 Technology Counts issue cites a 
number of people who tell us exactly what is 
going on in these schools. For example, Rob- 
ert F. McNergney, president of the American 
Association of Colleges of Education, tells us 
that colleges are still failing to blend training 
on integrating technology into the curricu- 
lum. He says, “Its not happening. Its 
extremely difficult to wedge technology train- 
ing into programs that are already packed.” 
And, of course, these colleges are under pres- 
sure to improve instruction across the board. 



New competency-based standards for school 
certification by the National Council for 
Accreditation of Teacher Education address 
curriculum as a whole as well as technology. 

The lack of preparation of teachers has 
ramifications beyond using technologies now 
in the classroom. The software teachers ask 
for, and the software they purchase, influence 
what the market supplies. While a lot of school 
buying is centralized, a lot is not. An Educa- 
tion Week survey found that while 23 states 
have group-purchasing programs, the general 
picture is one of teachers searching for soft- 
ware they think will help them teach; the ini- 
tiative rests with individual teachers, and tech- 
nology providers look to them to find out 
what they want and what will sell. Teacher 
development is not just preparing teachers for 
the technology now available; what becomes 
available depends on how much capability 
teachers possess. 

As standards-based reform becomes ubiq- 
uitous, teachers increasingly find themselves 
up against the blackboard to deliver what is 
being measured on standardized tests. If the 
curriculum has actually been changed to 
reflect new content standards, teachers will 
need technology to help them deliver that cur- 
riculum; and if tests are aligned to the revised 
curriculum, then the technology — if teach- 
ers find it and use it — will help their stu- 
dents do well on tests. But in many states, 
content standards become little more than 
specifications for new tests; there is no sys- 
tematic effort to change curriculum and 



Wenglinsky, H. ( 1 998). Does it Compute? The Relationship between Educational Technology and Student Achievement in 
Mathematics (ETS Policy Informarion Report). Princeton: NJ: Educational Testing Service. 



28 • Facing the Hard Facts 



31 



prepare teachers in its use. In such cases, teach- 
ers may well look for technology that is aligned 
with standardized tests, for that is where they 
will be judged. 

We tend to discuss technology as a sepa- 
rate matter, not as one piece of the education 
puzzle. But what the discussion above shows 
is that there can be a disconnect between con- 
tent standards, curriculum, technology, and 
tests. When this happens, technology may not 
be effective. Technology cannot make up for 
a standards-based reform effort that has fallen 
off track. Teachers might as well go back to 
the blackboard, an enduring technology. 

More and more information is being col- 
lected about the availability and use of tech- 
nology in the classroom — through surveys 
of the National Center for Education Statis- 
tics and private efforts, such as those of 
Education Week and the Milken Exchange on 
Education Technology. But a lot of key infor- 
mation is not available. We tend to count how 
many machines have been delivered to 
schools, and how many schools have been 
hooked up to the Internet. But we need to 
know more: How many of those computers 
are broken? How many are locked behind 
laboratory doors and seldom used? How are 
the Internet hook-ups being used in service 
of the curriculum? Perhaps we could devise 
some measure of computer-delivered instruc- 
tional hours, and then move on to identify- 
ing what kind of instruction these hours are 
used for. Perhaps it will take NAEP to lead 
the way, as it has over the years in other areas. 



There seem to be huge possibilities in 
harnessing technology to serve teaching and 
learning. Even with the nations attention riv- 
eted on education, we seem to have extraor- 
dinary difficulty taking advantage of it. While 
clear and steady progress is being made, the 
realization of the power of computers remains, 
in the words of a Charles Shultz character, an 
“insurmountable opportunity.” 
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X hi s report summarizes a broad spectrum of 
research on forces that affect student achievement. 



It draws on eight publications of the ETS 
Policy Information Center, as well as numer- 
ous other reports and sources. My concern 
has been to illuminate problems that are not 
being addressed, or have been addressed 
insufficiently, in the current education debate. 
I have attempted to glean from these many 
efforts and information sources those avenues 
to improving student achievement that are not 
being traveled — or at least not being given 
the emphasis they deserve — in the current 
education reform movement. In conclusion, 
I leave the following thoughts. 

Standards-based reform is in danger of 
becoming simply a testing movement; test- 
ing itself is not the treatment, but a way of 
finding out whether new content standards, 
rigorous curriculum, and teacher preparation 
are producing results. In a full standards-based 
reform effort, testing is just one important 
component. 

If there is no order in the classroom there 
will be no learning. Student behavior prob- 
lems are getting worse, not better; in the 
1990s, absenteeism, cutting classes, drug use, 
verbal abuse of teachers, and the presence of 
street gangs increased. While widespread 
attention has been given to murders at school, 
disruptive student behaviors have not been 
the focus of attention. Research shows that 
such behaviors do, in fact, adversely affect 



student achievement. There are, however, ef- 
forts in some places that can be emulated, and 
they are described in this report. 

While educational improvement sits at 
the top of opinion polls, family, student peers, 
and society send weak signals to students that 
high achievement is important. More than 
two out of five parents think extracurricular 
activities are as important as academic stud- 
ies; students fear that they will be less socially 
popular, if they work hard in school; employ- 
ers of high school graduates do not ask to see 
school transcripts; and many colleges have 
such low standards for admission that they 
routinely provide remedial courses. 

The nations focus has been largely on 
education policy and schools. But a learning 
policy that encompasses preschool develop- 
ment and draws on the resources of family 
and community is necessary to raise achieve- 
ment. Research clearly shows that, in addi- 
tion to school factors, family factors are 
important to learning. We need to better track 
sources of incentive for educational achieve- 
ment and learning, whether in school or out. 
While significant efforts are being made in 
many places, we must elevate such efforts to 
the level of attention given high content and 
performance standards if we are to enlarge 
young peoples' capacities and advance 
achievement to the high level the public 
is demanding. 
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The great promise of computers is not 
near to being realized. While hardware is 
becoming widely available, we are not yet put- 
ting education content into these machines 
and we must invest in the preparation of teach- 
ers to perform the miracles these machines 
may well be capable of. 

In identifying these areas where too little 
attention has been paid, I do not suggest that 
we lessen our efforts in implementing the stan- 
dards-based reform agenda; it makes good and 
common sense to make instruction rigorous, 
set high standards, and develop quality stan- 
dardized tests, the validity of which has been 
established for the purposes they are used. The 
quality of teacher preparation has drawn 
national attention, and important policy 
approaches are still being debated. However, 
drastic changes and considerable resources are 
needed, and there is little indication that 
the nation is prepared to do what needs to 
be done. 
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